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Abstract 

In this paper we continue our investigation on the high storage 
regime of a neural network with Gaussian patterns. Through an exact 
mapping between its partition function and one of a bipartite spin 
glass (whose parties consist of Ising and Gaussian spins respectively) , 
we give a complete control of the whole annealed region. The strategy 
explored is based on an interpolation between the bipartite system 
and two independent spin glasses built respectively by dichotomic and 
Gaussian spins: Critical line, behavior of the principal thermodynamic 
observables and their fluctuations as well as overlap fluctuations are 
obtained and discussed. Then, we move further, extending such an 
equivalence beyond the critical line, to explore the broken ergodicity 
phase under the assumption of replica symmetry and we show that 
the quenched free energy of this (analogical) Hopfield model can be 
described as a linear combination of the two quenched spin-glass free 
energies even in the replica symmetric framework. 

Introduction 

Neural networks, thought of as the harmonic oscillators of artificial intelli- 
gence, are nowadays being used in a huge number of different fields of science, 
ranging from practical application in data mining [13, 39j to theoretical spec- 
ulation in systems biology [2j [22], crossing fields as disparate as computer 
science |34| . quantitative sociology [8] or economics |19| . 
As a consequence, as applications develop, the need for mathematical meth- 
ods (bringing them under rigorous control) and a simple mathematical frame- 
work (acting as a benchmark for future speculation) increases and motivates 
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the present paper. 

Moreover, although the Hopfield model has been extensively studied since 
it was introduced in |33| , both from a physical [5j |6j [T^ |2"T] fM\ and a more 
mathematical [H [151 CES C3 ES EH EIJ H3] point of view, from the rigorous 
perspective many points about its properties remain unsolved, which also 
prompts further efforts in developing new mathematical techniques and dif- 
ferent physical perspectives. 

In the past, we gave an extensive treatment of an analogical neural network 
[9j[10j, namely a mean-field structure with N dichotomic neurons (spins) in- 
terconnected through Hebbian couplings [5j [21] whose p patterns are stored 
according to a standard Gaussian jV[0, 1]: In |9j we studied its thermody- 
namical properties paying attention to the annealed approximation (but we 
were unable, at that time, to gain a complete control of the whole annealed 
region), while in |10| we investigated the properties of the replica symmetric 
approximation. 

Within our approach, the equilibrium statistical mechanics of the neural net- 
work is shown to be equivalent to the one of a bipartite spin glass whose parts 
consist of the original N neurons (belonging to the first party, hence made 
of by dichotomic variables) and the other hand p Gaussians that give rise to 
the second part (hence consisting of continuous variables): As the theory of 
the mean-field Ising spin glass (namely the Sherrington Kirkpatrick model 
|35| ) has been intensively developed in the past decades (see for instance 
[3j|7j[28j|31j), while the same did not happen for the Gaussian counterpart, 
we investigated in detail the structure of the latter too, deepening the un- 
derstanding of its properties in 

Furthermore, to complete a streamlined description of the state of the art on 
this theme, we stress that results on the analogical Hopfield model, stemming 
from a mathematical perspective far from our connection with bipartite spin 
glasses, have also been obtained in |15 } I16| . 1X7] . 

Turning to the applied side, despite the fact that in neural networks (in their 
original artificial intelligence framework) the interest in continuous patterns 
is reduced or moved to rotators (e.g. Kuramoto oscillators [1]), as digital 
processing by Ising spins works as a better approximation for the standard 
integrate and fire models of neurons [18J, in several other fields of science 
(as, for instance, in chemical kinetics |22[ 123] or theoretical immunology [2]) 
continuous values of patterns can instead be preferred (|14]|20j) and a rig- 
orous mathematical control of completely continuous models (namely with 
both continuous patterns and neurons) belongs to our strategy of research. 
For the moment, we limit ourselves in presenting a clear scenario for the hy- 
brid model made of by continuous patterns and dichotomic variables, namely 
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the analogical neural network: In Section One we introduce the model and 
all the statistical-mechanics-related concepts. Then in Section Two we ex- 
pose our new strategy of interpolation which allows a complete control both 
of the ergodic region (confirming the annealed approximation, which is in- 
vestigated in great detail), and of the replica symmetric scenario, which is 
then deepened in Section Three. 
The last section contains our conclusions. 

Furthermore, an appendix is added: there the fluctuation theory of the order 
parameters of the model is discussed, and it is shown that the critical line 
found in this work characterizes a second order phase transition. 

1 The model, basic definitions and properties 
1.1 The analogical Hopfield model 

We introduce a large network of N two-state neurons (1, ..,N) 3 i — > di = 
±1, which are thought of as quiescent when their value is —1 or spiking when 
their value is +1. They interact throughout a synaptic matrix defined 
according to the Hebb rule for learning [32j [33] 



Each random variable £^ = {^, represents a learned pattern: While 

in the standard literature these patterns are usually chosen at random inde- 
pendently with values ±1 taken with equal probability 1/2, we chose them 
as taking real values with a unit Gaussian probability distribution, i.e. 



The analysis of the network assumes that the system has already stored p 
patterns (no learning is investigated here), and we will be interested in the 
case in which this number asymptotically increases linearly with respect to 
the system size (high storage level), so that p/N — > a as N — > oo, where 
a > is a parameter of the theory denoting the storage level. 
The Hamiltonian of the model has a mean-field structure and involves inter- 
actions between any pair of sites according to the definition 



p 




(1) 




(2) 




p 



N 



(3) 



/u=l i<j 
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1.2 Morphism in the bipartite model 

By splitting the summations Yld<j = 3 Yldj ~ \ $ij m the Hamiltonian 
([3]), we can introduce and write the partition function Z^ tP {f3;^) in the fol- 
lowing form 

ZN»m) = E-p(^EE^^-4EE(C) 2 ) (4) 

a ij {i=l i 

= «0e wE,1 = lE - lK,) 

where f3 > is the inverse temperature, and denotes here the level of noise 
in the network. We have defined 

a P N 

z n ,m = E ex p(^ E E tftfw)- ( 5 ) 

Notice that the last term at the r.h.s. of eq. does not depend on the 
particular state of the network, hence the control of the last term can be 
easily obtained [9] and simply adds a factor a/3/2 to the free energy. 
Consequently we focus just on Z(f3; £). Let us apply the Hubbard-Stratonovich 
lemma |25j to linearize with respect to the bilinear quenched memories car- 
ried by the CfCf ■ 
We can write 

zna^o = E/dl dz " ex % z " /2) ) ex p(^7^ E ( 6 ) 

For a generic function F of the neurons, we define the Boltzmann state ujr{F") 
at a given level of noise j3 as the average 

up(F) = u(F) = (Z N , P ((3; C))- 1 £ F(a)e-^(^) (7) 

and often we will drop the subscript /3 for the sake of simplicity. The s- 
replicated Boltzmann state is defined as the product state SI = uj 1 x u 2 x 
... x u s , in which all the single Boltzmann states are at the same noise level 
/3 _1 and share an identical sample of quenched memories £. For the sake of 
clearness, given a function F of the neurons of the s replicas and using the 
symbol a £ [1, .., s] to label replicas, such an average can be written as 

n(F(a\ ...,a s )) = ^ E E - E F ^ 1 ' ■•■> ctS ) ^(-z 3 E 0). 

W.P o-l 0-2 a s 0=1 

(8) 



4 



The average over the quenched memories will be denoted by E and for a 
generic function of these memories can be written as 

(n n *v )m = / f ^mo, (9) 

M =ii=i V^vr J 
with E[£f] = and E[(£f ) 2 ] = 1. 

Hereafter we will often denote the average over the gaussian spins as dfi(z). 

We use the symbol (.) to mean (.) = E$7(.). 

We recall that in the thermodynamic limit it is assumed 

lim ^- = a, 

N^oo N 

a being a given real number, which acts as free parameter of the theory. 

1.3 The thermodynamical observables 

The main quantities of interest are the intensive pressure, defined as 

lim A NtP (l3,0 = -(3 lim f N , p (P,0 = lim 1 In Z N , p (f3; £), (10) 

N-^oo TV— »oo N-+co iv 

the quenched intensive pressure, defined as 

lim A* NjP (f3) = -f3 lim f* N tf) = lim -j-ElnZ^O, (H) 

N— >oo ' r TV— »oo ' iV— >oo iv 

and the annealed intensive pressure, defined as 

lim A N M = -(3 lim f NjP (J3) = lim 1 lnEZ^ p (/3; £)■ (12) 

N— s-oo N— s-oo IV— s-oo iv 

According to thermodynamics, here /jv,p(/3,£) = v>N,p(/3,£) — f3~ 1 SN, P {(3,£,) is 
the free energy density, un, p (/3, £) is the internal energy density and sn, p (P, £) 
is the intensive entropy (the star and the bar denote the quenched and the 
annealed evaluations as well). 

According to the exploited bipartite nature of the Hopfield model, we in- 
troduce two other order parameters: the first is the overlap between the 
replicated neurons, defined as 



N 

1 

Qab 



if] of of €[-!,+!], (13) 



N 
i=i 
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and the second the overlap between the replicated Gaussian variables z, 
defined as 

1 P 

Pab = ~J2 Z »4 E (-° > +00 )- ( 14 ) 

These overlaps play a considerable role in the theory as they can express 
thermodynamical quantities. 



2 A detailed description of the annealed region 

2.1 The interpolation scheme for the annealing 

In this section we present the main idea of the work, used here to get a 
complete control of the high-temperature region: We interpolate between the 
neural network (described in terms of a bipartite spin glass) and a system 
consisting of two separate spin glasses, one dichotomic and one Gaussian. 
Note that, by the Jensen inequality, namely 

ElnZ NiP (p) <]nEZ NtP (p), 

we can write 

4v, p <ilnE^ f n^)^ Ei ^ r ^=ln2-^log(l-/3), (15) 

where we emphasize that the integral inside eq. (|15p exists only for (3 < 1. 
The N — > oo limit then offers immediately lini/v->oo A* N p (/3) < In 2 — a ln(l — 
/3)/2. The next step is to use interpolation to prove the validity of the 
Jensen bound in the whole region defined by the line f3 c = 1/(1 + yfa), 
which defines the boundary of the validity of the annealed approximation, 
in complete agreement with the well known picture of Amit, Gutfreund and 
Sompolinsky [5j|6j. 

To understand which is the proper interpolating structure, let us note that 
the exponent of the Boltzmann factor yields a family of random variables 
indexed by the configurations (a, z). For a given realization of the noise, 

H(a, z\t;) = \J^^2i^(,i,fj,criZ^ is a randomly centered variable with variance 
E(H(a, z\£)H(a', z'\£)) = ^ o-jcr-z^ = Ppqaa'Pzz'- 
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The presence of the product q a a'Pzz' hi the variance suggests the correct 
interpolating structure among this bipartite network and two other inde- 
pendent spin glasses, namely a Sherrington-Kirkpatrick model with variance 
q%. i and another spin glass model with Gaussian spin and variance p 2 zz i- It 
is in fact clear that a proper interpolating structure can be held by 

<p N (t) = ^Eln^ []Jd^)e^(ViJij2^i^) (16) 

a J fi=l 




exp ((1 - t)(—p zz —p zz )), 



where we have set 



2 

j 

ij 



and 

R ^ = "E Ji i ZiZ i 

ij 

and the average E is taken with respect to all the i.i.d. normal random 
variables £«, Jf/, Jy. The interpolation is performed such that for t = 1 
the interpolating structure (p(t = 1) returns the free energy of the bipartite 
model, namely of the neural network, while for t = it coincides with a 
factorization in an SK spin glass and a (suitably regularized) Gaussian one 
/3i, P^i which will be then fixed as opportune noise levels, for the moment 
are simply free parameters. 

As in [10j|30j, the plan is now to evaluate the flow under a changing t of the 
interpolating structure in order to get a positive defined sum rule by tuning 
opportunely /3i,/?2; hence, if we generalize the states as (.)t = Efij, where 
the subscript t accounts for the extended interpolating structure defined in 
(I16p we can write 

d -^r = ^/ 3 K^ ) *-^' fe ' )t )-i /3 K 1 -^' )t ) + (17) 

- j^/2(<pL>t - (pl')t) + ~Pl(pl)t - §£<p»>* (is) 

then, calling a = p/N even at finite size N (with a little language abuse), 
we can write 

^ = -f + + «/3 2 2 pL< " ^) ( - (19) 
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If we now impose on /3i,02 the constraint (3\(32 = \[a.f3 we get a perfect 
square in the brackets of the flow under a changing t, and calling St(a, f3) = 
(((3iq aa > — y/ctfi2Pzz') 2 )t the source term, we can write 

^>-\tf + S t (a,(3). (20) 
We can then integrate back between [0, 1] to get the following inequality 



1 f P IT 

<p N (l) = -Eln^y n^(^)eV^ E - C,Tl 



> 



lElnVe^/^ 
N ^ 



under the constraint fiifa = \/a(3. 

Note that K(a) in the above expression defines the SK-model, while the last 
term defines the regularized Gaussian spin glass deeply investigated in 
Now the advantages of this interpolation scheme become evident: As we have 
extremely satisfactory descriptions of the two independent models, namely 
the SK and the Gaussian spin glass, by these properties we can infer the 
behavior of the neural network (again thought of as the bipartite spin glass). 
In particular, we know that the free energies of each single part spin glass 
approach their annealed expression in the region where @i < 1 |41] and 
/3 + /32 < 1 Within this region, at the r.h.s. of eq. (|2T]l we get, in the 

thermodynamic limit, exactly In 2 — (a/2) ln(l — (3). 

Furthermore, if a and (3 respect the constraint (3(1 + y/ct) < 1, then finding 
/Si, ^2 such that the conditions (A), (B), (C) hold, being 

/3i/3 2 = v^/3 (A), fa<l (B), p + Pt<l (C), 

is certainly possible. In particular, using the SK critical behavior for the 
sake of simplicity, hence posing f5\ = 1, and setting (32 = \fa>(3, conditions 
(A) and (B) are automatically satisfied and, for the latter, being @2 = \Zct(3, 
we get 

(3 + (3 2 = (3 + V^(3 = (3(1 + v 7 ^) < 1, 
such that also condition (C) is verified. We can then state the following 
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Theorem 1. In the a,/3 plane there exist a critical line, defined by 



P c (a) = T — r , (21) 

i + V a 

such that for (3 < P c (a) the annealed approximation of the free energy holds 

i™^ Eln E / Ud^ei^^V = ln2-|ln(l-/3). (22) 
(j n 

Remark 1. We stress that the Borel-Cantelli lemma allows straightforwardly 
to determine the correct annealed regions for the SK model \J^1^ and, through 
a careful check of convergence of the integral defining the partition function, 
the same holds for the Gaussian case too JTTj ; however, the direct application 
of the Borel-Cantelli argument on the neural network gives a weaker result 
as shown for instance in JP^. The interpolation scheme allows to exploit and 
transfer the results for the SK and Gaussian models to the neural network, 
and enlarges the area of validity of the annealed expression for the free energy 
to the whole expected region, obtained e.g. via the replica method 



2.2 The control of the annealed region 

As a consequence, we can now extend the previous results exposed in [9] to 
the whole annealed region: Summarizing, we get the following 

Theorem 2. There exists /3 c (a), defined by eq. \21\). such that for f3 < /3 c (a) 
we have the following limits for the intensive free energy, internal energy and 
entropy, as N — )• oo and p/N — > a > 0: 

-P lim f NiP (P;0 = lim N' 1 In Z N>p (p;0 (23) 

= In 2- (a/2)ln(l - P) - (a/3/2), 
lim u N>p (P;0 = - lim N-%hiZ N>p (P;£) (24) 

= -a/3/(2(l-/3)), 
lim s NjP (P;0 = lim N~ x (In Z Nj ,{P\Z)- pdp In Z NjP (P;0) (25) 

N— >co N— >oo 

= In 2 - (a/2) ln(l - P) - (a/? 2 )/(2(l - P)) - (a/3/2), 

^-almost surely. The same limits hold for the quenched averages, so that in 
particular 

lim N^ElnZ^iP^) = In 2 - ^ ln(l - p) - 
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where, in all these formulas, the last term, namely —a/3/2, arises due to the 
diagonal contribution of the complete partition function |^). 

Theorem 3. There exists (3 c (a), defined by eq. \21\). such that for {3 < f3 c {a) 
we have the following convergence in distribution 

lnZ NtP (P;0-lnEZ N>p (/3;0 -»• C(fi) + X S(P) (26) 

where \ i- s a unit Gaussian in M[0, 1] and 



C(fi) = --lnVl/(l-^ 2 /3 2 «) (27) 



i 



5(/3) = (l nx /l/(l-^/3 2 a)) 5 , (28) 
with a = (1- 0)- x . 

3 Extension to the replica symmetric solution 

Once the correct interpolating structure is understood, and spurred by the 
observation that the replica symmetric expression for the quenched free en- 
ergy of the three models, namely the analogical neural network, the SK spin 
glass and the Gaussian one, are well known and investigated (for instance 
in |30j[7j[26j[9j[llJ) we want to push further the equivalence among neural 
network and spin glasses, giving a complete picture also of the replica sym- 
metric approximation. 

To this task, let us recall that the replica symmetric approximation of the 
quenched free energy of the analogical neural network A^f N (a, /3) is given by 
the following expression [9| 

Affiffap) = In 2 + y dfi(z) In cosh(z^/aj3~p) + | ln( - _ ^ _ -) ) + 

a/5 q a/3 

+ T i-^i-j) -^ 1 -^ (29) 

where the order parameters denoted with a bar (to mean their RS approxi- 
mation) are given by 



P 



dfi(z) tanh 2 (zy/ a.f3p*j , (30) 
Pq/(l-P(l-q)y. (31) 
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Let us introduce further j3\ and 2 as 

02 = l-f3(l-q), (33) 
such that j3i 02 = y/a/3. We need also the RS approximation Ag^(0i) of the 
quenched free energy of the SK model, at the noise level 0i, namely 

Af K ((5i) = In 2 + J dfi(z) lncosh(/? 1 v&) + - g SJf ) 2 , (34) 

where 

95^ = y <fyt(*) tanh 2 {PxZy/qsicj ■ (35) 

By a direct comparison among the overlap expressions (|30[ 135]) we immedi- 
ately conclude that we must have 

PlQSK = a/3p, 

which indeed holds as it can be verified easily, bearing in mind the expression 
(I3T1) and (J32|) for p and X . 

As a last ingredient we need to introduce also the replica symmetric expres- 
sion Aq^ uss (02, $) °f the Gaussian spin glass at a noise level 02 as |llj 

A%Ls(P2,P) = \\na+ l -0lp G a 2 + ^0 2 p 2 G , (36) 

where 

PG = (/3 2 -(l-/?))//3|, (37) 

a 2 = 1/(1- + 2 p G ). (38) 

Note that the definition of the overlap between continuous variables encoded 
by eq. (|3ip is in perfect agreement with the same overlap defined within the 
framework of eq. (i3Tl) . because, being 02 = 1 — P{\ — 0), we can write 

_ 02 ~ (1 ~ P) _ l-0{l~ q) ~(l-0) _ 0Q ,„ q , 

PGauss 0\ (\-0{\-q)Y (l-0(l-q)y 1 > 

As a consequence, through a direct verification by comparison (that we omit 
as it is long and straightforward), we can state the final theorem of the paper: 
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Theorem 4. Fixed, at noise level (3, f3% and fa as in h32\) and h33\) . the 
replica symmetric approximation of the quenched free energy of the analogical 
neural network can be linearly decomposed in terms of the replica symmetric 
approximation of the Sherrington- Kirkpatrick quenched free energy, at noise 
level fa , and the replica symmetric approximation of the quenched free energy 
of the Gaussian spin glass, at noise level fa, such that 

Aff N (P) = Af K (fa) - lp\ + aA Gauss (fa,/3), (40) 

and the inequality \21\) becomes an identity for the RS behavior. 

Remark 2. We stress that the above Theorem is in agreement with the sum 
rule t20\) of Section 2 as, in the replica symmetric approximation, q aa i = q 
and p zz / = p, hence 

M-A'pf^^Mpl^-O. (41) 

Remark 3. Approaching the high-temperature region we have q — > and 
p — >• 0, and clearly f3 —> 1/(1 + yfa). As a consequence we have 

P 2 = 1-/3(1 1-1/(1 + ^), (42) 

ft - - m 

then f5 + ^2 1? such that also the single-party counterparts approaches their 
critical points. 

Coherently, inside the annealed region we get q = 0, then with the ex- 
pressions for f3\ , 02 we can write $2 + P = 1 that is the boundary of the 
annealed region for the Gaussian spin glass, while /3\ = y/a 13/(1 — (3) be- 
cause (3 < 1/(1 + yfa) we get (3\ < 1, namely the annealed region of the SK 
model. 



4 Conclusions and Outlook 

Neural networks are becoming the paradigm of a wide family of complex 
systems with cognitive capabilities such as memory and learning both in the 
living world and outside. 

As a consequence, a solid control of these networks is fundamental: In this 
paper we provided a clear analysis of the analogical neural network thought 
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of as a bipartite spin glass, made of by two different type of spins: one en- 
semble of dichotomic variables, as in the celebrated Sherrington-Kirkpatrick 
model, and one ensemble made of by Gaussian distributed variables. 
Exploiting this analogy, we developed a new interpolation scheme among 
the bipartite spin glass that mirrors the neural network and two indepen- 
dent glassy systems. Through this novel technique, we have then shown how 
to get a complete control of the annealed region of the neural network: The 
critical line has been obtained, together with an explicit behavior of all the 
main thermodynamical quantities: free energy, internal energy, entropy and 
overlaps (namely the order parameters of the theory) . 

One step forward we extended our interpolation scheme beyond the ergodic 
region, under the assumption of replica symmetry: We showed that the 
replica symmetric approximation of the quenched free energy of the ana- 
logical Hopfield model (at noise level (3) can be expressed in terms of the 
replica symmetric expressions of the quenched free energies of the SK model 
(at noise level f3\) and of the Gaussian model (at noise level ^2), and we 
obtained the equations linking /3, /3i,/?2 obtaining then a complete control 
also within this framework. 

All that opens very interesting perspectives. The structure of the neural 
network as a linear combination of spin glasses is very rich: in fact we know 
that, as the SK model presents a very glassy full RSB structure [28], in the 
Gaussian one this is absent, since the true solution is in fact RS even with no 
external field Thus one could aspect in our analogical neural network a 
competition of these two effects: rather a new feature in the complex systems 
scenario, that has to be deeply investigated. 

Clearly we would deepen this topic, for example within a fully broken replica 
symmetry scenario on which we plan to report soon. 

Furthermore, the analogical model shares many features with the original 
Hopfield model (which is even harder from a mathematical point of view) 
for which one could study in what measure this structure is preserved. 
Future outlooks should cover also the completely analogical model in order 
to develop mathematical techniques beyond the standard ones required in 
artificial intelligence and closer to system biology. 
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Appendix. 

Fluctuation Theory for the Order Parameters 



We develop in this appendix a fluctuation theory of the order parameters 
to see that the ergodicity breaking is accomplished through a second order 
phase transition (i.e. the overlap fluctuations, properly rescaled over the 
volume, do diverge on the line /3 c (ai) hence defining a critical phenomenon). 
To satisfy this task we proceed as follows: at first we introduce a different 
interpolating structure with respect to the one discussed above (developed 
and discussed in [10J) to bridge the neural network with two single party 
one-body models where spins are subjected to random fields in a way close 
to stochastic stability [40j or cavity perspective |29| . Then we evaluate the 
flow with respect to the interpolating parameter so to be able to calculate 
variations of generic observable as overlap correlation functions. 
Then we define the centered and rescaled overlaps and introduce their cor- 
relation matrix. Each element of this matrix then is evaluated at t = and 
then propagated thought t = 1 via its flow: This procedure encodes naturally 
for a system of coupled linear differential equations that, once solved, gives 
the expressions of the overlap fluctuations. The latter are found to diverge 
on the critical line /3 c (a) already outlined and this will close our inspection 
of the annealed regime. 

Let us start the plan by introducing the next interpolating structure: 
In a pure stochastic stability fashion |10] . we need to introduce also two 
classes of i.i.d. JV[0, 1] variables, namely N variables r\i and p variables fja, 
whose average is still encoded into the E operator and by which we define 
the following interpolating quenched pressure <pN p (/3,t) 



We stress that t G [0, 1] interpolates between t = where the interpolating 
quenched pressure becomes made of by non-interacting systems (a series of 
one-body problem) whose integration is straightforward (as well as the evalu- 
ation of the overlap correlation functions it produces) and the opposite limit, 
t = 1, that recovers the correct quenched free energy. Then we can evaluate 




where 



a = 





14 



the flow with respect to the Boltzman factor encoded in the structure (|44p 
as stated in the next 



Proposition A. Given O as a smooth function of s replica overlaps (qi, q s ) 
and (pi, ...,p s ), the following streaming equation holds: 

1 s 

-{0) t = /3^(^<0-£a, b r/ a , b > t (45) 

a,b 

- s • fa,a+l*7a,s+l)t H g (O ■ 6+l,«+2»7s+l )S +2)tJ • 

a=l 

We skip the proof as it is long but simple and works by a direct evaluation 
which is pretty standard in the disordered system literature (see for example 

mm)- 

The rescaled overlap £12 and 1712 are defined accordingly to 

£12 = ^(312-?), (46) 
n 12 = Vk(p\2 ~ Pj • (47) 

In order to control the overlap fluctuations, namely (£i 2 )t=i, (£12^12)4=15 
{r/i 2 )t=i, •■■) noting that the streaming equation pastes two replicas to the 
ones already involved (s = 2 so far), we need to study nine correlation 
functions. It is then useful to introduce them and link them to capital 
letters so to simplify their visualization: 



(&)* = A(t), (£ 12 £ 13 ) t 

(£12*712)* = D(t), (£12*713)* 

(7712*712)* = G (*)> (*7l2*7l3)t 

If we introduce the operator dot as 



= B(t), (£ 12 £ 34 ) t = C(t), (48) 
= E(t), (£12*734)* = F(t), (49) 
= H(t), (rnam)t = I(t). (50) 



f3\J~OL dt 

which simplifies calculations and shifts the propagation of the flow from t = 1 
to t = fiyfa. Assuming a Gaussian behavior, as in the strategy outlined in 
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|30| . we can write the overall flow of the overlap correlation functions in the 
form of the following differential system 



A = 2AD — 8BE + GCF, 

B = 2AE + 2BD - ABE - QBF - QEC + 12CF, 
C = 2AF + 2CD + 8BE - 16BF - 16CE + 20CF, 
D = AG - ABB + 3CI + D 2 - AE 2 + 3F 2 , 

E = AH + BG - 2BH — 3BI — 3CH + 6CI + 2ED — 2E 2 — 6EF + 6F 2 , 
F = AI + CG + 4BH — 8BI — 8CH + IOC I + 2DF + AE 2 — IQEF + 10F 2 , 
G = 2GD-8HE + 6IF, 

H = 2GE + 2HD-AHE-6HF-6IE + 12IF, 
I = 2GF + 2DI + 8HE - 1QHF - 161 E + 20IF. 

Although it may appear complex, it is relatively easy to solve this system, 
once the initial conditions at t = are known (information then can be 
obtained straightforwardly as at t = everything factorizes the theory being 
one-body). Our general analysis covers also the case where external fields 
are involved. We do not report here the full analysis, for the sake of brevity. 
Here, as we are interested in finding where ergodicity becomes broken, we 
start propagating t 6 — > 1 from the annealed region, where q = and 
p = 0, which simplifies further the problem: 

In fact, it is immediate to check that, for the only terms that we need to 
consider, A,D,G (the other being strictly zero on the whole t £ [0,1]), the 
starting points are A(0) = 1, D(0) = 0, G(0) = (1 — /3)~ 2 and their evolution 
is ruled by 



The solution of this differential system is long but straightforward then we 
skip the proof and directly state the next 

Theorem A. In the ergodic region the behavior of the overlap fluctuations 



A = 2AD, 
D = AG + D 2 



G 



2GD. 



(51) 
(52) 
(53) 
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is regular and described by the following equations 

<&> = (iV-W (54) 

= (1-/3)2-^2^ (56) 

diverging on the critical line /3 c (a), defined by eq. 1121]). hence defining a 
second order phase transition. 
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